1. Field of the Invention
The present invention relates to an image processing apparatus for obtaining a broader-range photograph by photographing a subject image in a plurality of divided parts and joining or composing together part-images.
2. Description of the Related Art
Generally, an image pickup apparatus using a solid image pickup element, such as a CCD, has been broadly employed for electronic still cameras, video cameras, etc., and there has been a growing demand for those having a higher resolution and for those having a broader area range.
A technique for achieving a broader range photographing as well as a higher image resolution has been disclosed in U.S. patent application Ser. No. 08/045,038 (abandoned in favor of File Wrapper Continuation Serial No. 08/969,937) filed by the applicant. For the purpose of achieving a higher-resolution apparatus, a proposal has been made there to obtain a composed image by photographing an image with one photographing lens, picking up the image in divided parts by a plurality of image pickup elements and subjecting those part-images to signal processing. Also, a proposal has been made there to obtain a composed image by photographing a subject image in divided parts through the switching of the light beam by a mirror and through the switching of a fixed area by a camera/subject movement and subjecting the corresponding data to signal processing, noting that, in this case, use is made of only one image pickup element. Both are common in terms of the signal processing as will be explained below with respect to FIGS. 23 to 25.
FIG. 23 is a view showing a whole arrangement of a conventional image pickup apparatus.
In the image pickup apparatus of FIG. 23, image inputting sections 1a to 1c, each, comprise a photographing lens 59, image pickup section 60, such as a CCD, and A/D converter 2 and are so arranged that a subject image is overlappingly taken in different positions. For brevity's sake, here, a signal involved is of a monochrome type.
Here it is assumed that the subject 6 is a planar document such as a design drawing. The outputs of the image pickup sections are A/D converted by the respective A/D converters 2 to digital equivalents and the outputs of the A/D converters 2 are input to an image composing section 3, as images a, b and c. Through the later-described processing at the image composing section 3 a composed image is generated as images a, b, c, as shown in FIG. 24 and is output to a monitor 4 and to a printer 5.
The image composing section 3 detects a positional image-to-image displacement from those signals corresponding to overlapped areas in an image, interpolates the image in accordance with the image areas and provides an image-to-image seam.
FIG. 25 shows an arrangement of the image pickup section 3.
The image composing section 3 comprises frame memories 7a to 7c for storing the images a, b, c, displacement detector 8a for detecting a positional displacement of the images a, b as a parallel movement amount S1 and rotational amount R1 from those signals in the overlapped areas of images a, b, displacement detector 8b for detecting a positional displacement of images b, c as a parallel movement amount S2 and rotational amount R2 from those signals corresponding to the overlapped areas of the images b, c, interpolation calculator 9b for performing interpolation calculation by the use of an affine transformation so that the image b is joined to an array for the image a on the basis of S1, R1, interpolation calculator 9c for performing interpolation calculation so that the image c is joined to an array for the images a, b on the basis of S2, R2, multiplies 11a to 11c which, in order to perform processing for making an image-to-image seam less pronounced, effecting multiplication by coefficient Ca, Cb, Cc so that these coefficients are gradually varied (an example between given two points P, Q) at the overlapped areas in the image as shown in FIG. 24, coefficient setting units 10 for setting these coefficients, adders 12 for additively combining the outputs of the multipliers 11a to 11c, and a frame memory 13 for storing a joined image.
By the operation of the image composing section 3 it is possible to obtain a better composed image even if there are some rotations in the image as shown in FIG. 24.
Although the conventional arrangement has been explained as using three cameras, the same is true about two or four or more cameras.
In the above-mentioned conventional image pickup apparatus, an image is composed through the parallel movement and rotation and, if there is aberration such as a distortion in the photographing lens, an image varies in the overlapped areas at a time of simple parallel movement and rotation. This involves a disadvantage in that, due to the failure to properly detect the positional displacement and due to the image differing at those overlapped areas after being interpolated, an image emerges in very unsightly way.
It has been common practice to mount the electronic camera, etc., on a zoom lens when it is employed. When a grid-like pattern is to be photographed, a barrel-shaped distortion aberration is generally produced, in particular, on a wide-angle side, as shown in FIG. 26A. In such an image example, an image varies in the overlapped areas and hence an image composition is difficult to achieve.
In the case where a three-dimensional subject, not the planar document, is to be photographed, parallax occurs in a plurality of cameras and the same problem as set out above occurs.
FIG. 27 shows the case where objects in varying distances are photographed. Since an object (a table) on the front side varies in shape due to the effect of parallax, it suffers the same adverse effect as in the case of aberration.